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(57) Abstract: In a speech recognition system of the type adapted to process utterances from a caller or user by way of components 
including a recogniser, an utterance processing system and a dialogue processing system so as to produce responses to said utterances, 
a method of parsing a data structure derived by a component of said system; said method comprising performing only a partial parse 
of said data structure. 
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STOCHASTIC CHUNK PARSER 
The present invention relates to a stochastic chunk 
parser and, more particularly, to such a parser suited for 
use within an automated speech recognition system. 

5 

BACKGROUND 

Automated speech recognition is a complex task in 
itself. Automated speech understanding sufficient to 
10 provide automated dialogue with a user adds a further layer 
of complexity. 

In this specification the term u automated speech 
recognition system" will refer to automated or 
substantially automated systems which perform automated 
15 speech recognition and also attempt automated speech 
understanding, at least to predetermined levels sufficient 
to provide a capability for at least limited automated 
dialogue with a user. 

A generalized diagram of a commercial grade automated 
20 speech recognition system as can be used for example in 
call centres and the like is illustrated in Fig. 1. 

With advances in digital computers and a significant 
lowering in cost per unit of computing capacity there have 
been a number of attempts -in -the -commercial- marketplace to 
25 i install such automated speech recognition . systems fe 
implemented substantially by means of digital computers. 
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However, to date, there remain problems in achieving 
100% recognition and/or 100% understanding in real time. 

In particular parsing techniques are used to help take 
utterances input to the system so as to decide the 
5 likelihood of different interpretations being placed on 
those utterances. 

Parsing techniques that produce complete parse trees 
and attribute value matrices have been used. Also, parsing 
techniques have been used which produce, complete parse 
10 trees but may have skipped over one or more words. The 
grammars used in these solutions have been difficult to 
develop and maintain. 

These problem has not been satisfactorily solved in 
the past due to the complicated nature of spoken language, 
15 containing, for example, repairs, pauses, discourse 
markers, asides, editing terms, etc. In addition, many- 
words in spoken language are ignored by the application but 
are required to be included in grammars used by parsers 
searching for complete parsers. 
20 It is an object of the present invention to address or 

ameliorate one or more of the abovementioned disadvantages. 

BRIEF DESCRIPTION OF INVENTION 
25 Accordingly, in one broad form of the invention there 

is provided in a speech recognition system of the type 
adapted to process utterances from a caller or user by way 
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of components including a recogniser, an utterance 
processing system and a dialogue processing system so as to 
produce responses to said utterances, a method of parsing a 
data structure derived by a component of said system; said 
5 method comprising performing only a partial parse of said 
data structure. 

In a further broad form of the invention there is 
provided in a speech recognition system of the type adapted 
to process utterances from a caller or user by way of 
10 components including a recogniser, an utterance processing 
system and a dialogue processing system so as to produce 
responses to said utterances, a stochastic chunk parser for 
attribute grammars which searches for meaningful words and 
phrases within said utterances and from these builds 
15 attribute value matrices. 

Preferably said parser utilizes structure and location 
of said phrases to estimate the probability of the 
utterance in which they are found. 

Preferably said parser receives as input an N-best 
20 list derived from said recogniser. 

Preferably said attribute value matrices attach 
respective probabilities to respective attribute value 
matrices by combining phrase structure probabilities with 
class-based N-gram language modeling. 
25 Preferably said method is applied to a data structure. 
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Preferably said data structure comprises a N-best 

list. 

Preferably said data structure comprises a word 

I 

I^JLattice. 
5 

BRIEF DESCRIPTION OF DRAWINGS 

Embodiments of the present invention will now be 
described with reference to the accompanying drawings 
10 wherein: 

Fig. 1 is a generalized block diagram of a prior art 
automated speech recognition system; 

Fig. 2 is a generalized block diagram of an automated 
speech recognition system suited for use in conjunction 
15 with an embodiment of the present invention; 

Fig. 3 is a more detailed block, diagram of the 
utterance processing and dialogue processing portions of 
the system of Fig. 2; 

Fig. 4 is a more detailed block diagram of the system 
20 of Fig. 2 incorporating a stochastic chunk parser in 
accordance with a first preferred embodiment of the present 
invention. 



DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
25 With reference to Fig. 2 there is illustrated a 

generalized block - diagram of an automated speech 
recognition system 10 adapted to receive human speech 
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derived from user 11 , and to process that speech with a 
view to recognizing and understanding the speech to a 
sufficient level of accuracy that a response 12 can be 
returned to user 11 by system 10. In the context of 
5 systems to which embodiments of the present invention are 
applicable the response 12 can take the form of an auditory 
communication, a written or visual communication or any 
other form of communication intelligible to user 11 or a 
combination thereof. 

10 In all cases input from user 11 is in the form of a 

plurality of utterances 13 which are received by transducer 
14 (for example a microphone) and converted into an 
electronic representation 15 of the utterances 13 . In one 
exemplary form the electronic representation 15 comprises a 

15 digital representation of the utterances 13 in .WAV format. 
Each electronic representation 15 represents an entire 
utterance 13 . The electronic representations 15 are 
processed through front end processor 16 to produce a 
stream of vectors 17, one vector for example for each 10ms 

20 segment of utterance 13. The vectors 17 are matched 
against knowledge base vectors 18 derived from knowledge 
base 19 by back end processor 20 so as to produce ranked 
results 1-N in the form of N. best results: 21. The results 
can comprise for example subwords, words or phrases but 

25 will depend on the application. N.can vary from 1 to very 
high values, again depending on the application. 
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An utterance processing system 26 receives the N best 
results 21 and begins the task of assembling the results 
into a meaning representation 25 for example based on the 
data contained in language knowledge database 31. 
5 The utterance processing system 26 orders the 

resulting tokens or words 23 contained in N-best results 21 
into a meaning representation 25 of token or word 
candidates which are passed to the dialogue processing 
system 2 7 where sufficient understanding is attained so as 

10 to permit functional utilization of speech input 15 from 
user 11 for the task to be performed by the automated 
speech recognition system 10. In this case the 

functionality includes attaining of sufficient 

understanding to permit at lease a limited dialogue to be 

15 entered into with user/caller 11 by means of response 12 in 
the form of prompts so as to elicit further speech input 
from the user 11. in the alternative or in addition, the 
functionality for example can include a sufficient 
understanding to permit interaction with extended databases 

20 for data identification. 

Fig. 3 illustrates further detail of the system of 
Fig. 2 including listing of further functional components 
which make up the utterance processing system 2*6 and the 
dialogue processing system 27 and their interaction. Like 

25 components are numbered as for the arrangement of Fig. 2. 
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The utterance processing system 26 and the dialogue 
processing system 27 together form a natural language 



event -driven and processes each of the utterances 13 of 



system 2 7 puts any given utterance 13 of caller/user 11 
into the context of the current conversation (usually in 
the context of a telephone conversation) Broadly, in a 
telephone answering context, it will try to resolve the 

10 query from the caller and decide on an appropriate answer 
to be provided by way of response 12 . 

The utterance processing system 26 takes as input the 
output of the acoustic or speech recogniser 3 0 and produces 
a meaning representation 25 for passing to dialogue 

15 processing system 27. 

In a typical, but not limiting form, the meaning 
representation 25 can take the form of value pairs. For 
example, the utterance "I want to go from Melbourne to 
Sydney on Wednesday" may be presented to the dialogue 

20 processing system 27 in the form of three value pairs, 
comprising .- 



proc e s s i ng sy s t em . 



The utterance processing system 2 6 is 



5 caller/user 11 individually. 



The dialogue processing 



1. 



Start; Melbourne 



2. 



De s t ina t i on ; Sydney 



3 . 



Date; Wednesday 
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where, in this instance, the components Melbourne, Sydney, 
Wednesday of the value pairs 24 comprise tokens or words 
23. 

With particular reference to Fig, 3 the recogniser 3 0 
5 provides as output N best results 21 usually in the form of 
tokens or words 23 to the utterance processing system 2 6 
where it is first disambiguated by language model 32. In 
one form the language model 32 is based on trigrams with 
cut off . 

10 Analyser 3 3 specifies how words derived from language 

model 32 can be grouped together to form meaningful phrases 
which are used to interpret utterance 13. In one form the 
analyzer is based on a series of simple finite state 
automata which produce robust parses of phrasal chunks - 

15 for example noun phrases for entities and concepts and WH- 
phrases for questions, dates. Analyser 33 is driven by 
grammars such as meta- grammar 34. The grammars themselves 
must be tailored for each application and can be thought of 
as data created for a specific customer. 

20 The resolver 35 then uses semantic information 

• associated with the words of the phrases recognized as 
relevant by the analyzer 33 to refine the meaning 
representation 25 into its : . finals .form for : passing . through 
the dialogue flow controller 3 6 within dialogue processing 

25 system 27. 
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The dialogue processing system 27, in this instance 
with reference to Fig. 3, receives meaning representation 
25 from resolver 35 and processes the dialogue according to 
the appropriate dialogue models. Again, dialogue models 
5 will be specific to different applications but some may be 
reusable. For example a protocol model may handle 

greetings, closures, interruptions, errors and the like 
across a number of different applications. 

The dialogue flow controller 36 uses the dialogue 
10 history to keep track of the interactions. 

The logic engine 37, in this instance, creates SQL 
queries based on the meaning representation 25. Again it 
will be dependent on the. specific application and its 
domain knowledge base. 
!5 The generator 3 8 produces responses 12 (for example 

speech out) . In the simplest form the generator 3 8 can 
utilize generic text to speech (TTS) systems to produce a 
voiced response . 

Language knowledge database 31 comprises, in the 
20 instance of Fig. 3, a lexicon 39 operating in conjunction 
with database 40. The lexicon 39 and database 40 operating 
in conjunction with knowledge base mapping tools 41 and, as 
appropriate, language model -32 .and . grammars 34' -constitutes . 
a language knowledge database 31 or knowledge base which 
25 deals with domain specific data. The structure and 

grouping of data is modeled in the knowledge base 31. 
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Database 4 0 comprises raw data provided by a customer. 
In one instance this data may comprise names, addresses , 
places, dates and is usually organised in a way that 
logically relates to the way it will be used. The database 
5 40 may remain unchanged or it may be updated throughout the 
lifetime of an application. Functional implementation can 
be by way of database servers such as MySQL , Grade, 
Postgres . 

As will be observed particularly with reference to 
10 Fig. 3, interaction between a number of components in the 
system can be quite complex with lexicon 3 9 , in particular, 
being used by and interacting with multiple components of 
System 10. 

With reference to Fig. 4 a stochastic chunk parser 
15 1310 in accordance with a first preferred embodiment of the 
invention is illustrated in block diagram form. Parser 
1310 takes as its input the output of recogniser 3 0 and, 
with the assistance of a phrase structure grammar 1311 it 
produces a set 1312 of attribute value matrices, each 
20 having associated with it a probability 1314. In this 
instance the set 1312 comprises N attribute value matrices 
1313 together with a corresponding respective N 
probabilities 1314. in. one .particular/, form - the 

probabilities are calculated by combining phrase structure 
25 probabilities with class-based N-gram language modeling, 
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thereby to allow the calculation of probabilities of 
partial parses* 

The Stochastic Chunk Parser for Attribute Grammars 
searches for meaningful words and phrases, and from these 
5 builds attribute value matrices (AVM) . It also uses the 
structure and location of these phrases to estimate the 
probability of the utterances they are found in. 

The solution differs from previous attempts in that no 
complete parse of the sentence is performed. This means 
10 the parser, does not attempt to find structural analysis for 
repairs, discourse markers, asides, editing terms, and 
other words which do not contribute to the construction of 
AVMs. 

The Stochastic Chunk Parser for Attribute Grammars 
15 1310 takes as its input a scored N-best list 21 created by 
a speech recogniser 30. From this, the Parser uses partial 
parsing to create a set of AVMs. The parsing is 
stochastic, so that probabilities are associated with each 
AVM. This is used to calculate the most likely AVM from 
20 the speech input . 

The Parser 1310 uses phrase structure grammars 1311, 
and proceeds from the bottom up. As each phrase structure 
rule is matched AVMs 1313are created representing the 
meaning of the phrase. At the end of structural analysis 
25 there is a sequence of terminal and non- terminal 
categories; the former represent unparsed words, the latter 
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parsed structures. The probabilities of these sequences are 
calculated in a novel manner, combining phrase structure 
probabilities with class-based n-gram language modelling. 
The internal structure of phrases are calculated using 
5 probabilistic parsing techniques, and the product of these 
is multiplied with the probability of the sequence of 
categories, which is estimated using an N-gram language 
model . 

The same principle can be used to do parsing on data 
10 structures other than N-best lists, for example word 
lattices. 

The advantage of the solution is that allows the 
robust stochastic parsing of spoken language. The grammars 
required by the Parser are considerably simpler than those 
15 used by parsers doing complete analyses . This means that 
application development time is significantly reduced. 

The principle underlying the solution is that of using 
a mixture of phrase probabilities and N-gram language 
modelling to allow the calculation of probabilities of 
20 partial parses. 

Variations that embody the concept include other 
methods of combining phrase structure probabilities with 
the probability of a series of tokens, including fixed and 
variable length language models, word based language models 
25 that backoff to ciass based, etc. 
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Annexure A provides additional background on the 
concept of probabilistic partial parsing and is 
incorporated herein by cross reference and so as to form 
part of this specification. 
5 The above describes only some embodiments of the 

present invention and modifications, obvious to those 
skilled in the art, can be made thereto without departing 
from the scope and spirit of the present invention. 
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Probabilistic Partial Parsing 

Introduction 

Probabilistic Parsing assigns probabilities to parse trees. This has two possible functions in the Field of 
spoken language understanding: 

L. Re-scoring speech recognition candidates. 
2. Choosing between ambiguous parses. 

The first function addresses what has been a concern for some lime: how can linguistic knowledge be used 
to improve recognition performance? Language modelling using N-grams is one way to incorporate 
statistics on language use* but it is unable to sensibly cope with syntactic relations other than the most local. 

To dale, 2. has not been an issue for use since our parser has been deterministic. Recall that a deterministic 
parser producesd one and only one parse for each sentence, while a non-deterministic parser is capable of 
producing multiple parses for a sentence. Note mat since we are doing partial parsing we are always 
guaranteed of finding a partial parse for a sentence, however "partial" may be "zero", in which case no 
parsing whatsoever may have been done to produce the partial parse. While deterministic parsers are the 
fastest, and to date have been sufficient for our purposes, it should be kept in mind that non-deterministic 
parsing approaches may be required in the future 

Non-deterministic parsing 

In this section I give an example where non-deterministic parsing is required to produce the desired 
analysis. (Note that the example is meant to be illustrative only - the problem could also be solved by 
including **turn_off * in the lexicon.) Consider the following two verb phrases. 

(1) turn off the light 

(2) turn off the road 

The correct chunking off these would be 1 : 

(3) (turn off) (the light) 

(4) (turn) (off (the road)) 

Consider the following analysis grammar; 

Level 1 

VERB -> VB 

then Action = turn 
end. 

PI IRAS ALVBRB -> VB IN 

if I v eq turn and 2.v eq off 
then Action = extinguish 
end. 

NP -> 1 OT | NN 

then Object = 2.v 
end. 



1 According to the analysis in Rodney Huddlcsionc's I m rod u a. mi h, the Grammar of English. 
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Level 2 

PP IN NP 

if Lv eq off and 2.0bject eq road 
then DirectionOff = 2-Object 
end. 

The above grammar will produce the parses: 

(5) * (turn) off (the light) 

(6) (turn) (off (the road)) 

If the precedence of the first two rules is swapped, the grammar will instead produce 

(7) (lum off) (the light) 

(8) * (turn off) (the road) 

No re-ordering of the rules will produce the correct chunking of both verb phrases. This is because the 
correct Level 1 chunking of turn is co-dependent with the correct chunking of the complement phrase. One 
solution lo this would be to introduce context-sensitivity into the grammar, but this is undesirable. There 
are strong arguments lhat it makes sense to keep both (fum ) off (the road) and ((urn off) (the road) alive as 
candidate IjcvcI 1 chunkings until the applicability of Level 2 rules has been checked. 

To summarise: 

• Probabilistic parsing allows choosing amongst competing candidate parses; 

• Producing competing candidate parses requires non-deterministic parsing. 

J have tried to provide a brief sketch of bow non-deterministic parsing may be required when we attempt 
more complicated semantic analyses. I will now discuss interesting issues related to non-deterministic 
partial parsing. 

Non-dctcrministic parsing and lattices 

Non-determinism parsing has an interesting connection with lattices that is worth discussing. In order to 
keep future possibilites open, non-deterministic parsing essentially treats all grammar rules as oplional: a 
rule may be applied, or it may be skipped in order to allow future rules the possibility of applying. Since 
this leads lo a binary branching of the search space, il may be supposed that this leads to an exponential 
explosion of the search space. This belief is misguided however, since the interdependence of rules, both at 
the same level, and across levels, places a large restriction on the number of parses. Furthermore, storing 
possible hypotheses in a chart allows efficient searching for possible parses. For example, considering the 
grammar above as non-deterministic, the following parse for turn off (he road would be constructed. 





PP: f'-thc. road 


I>J JRASALYHRB: (urn off 




V1£RB: turn 




NP: (he road 


VB 


IN 


DT . 


NN 


turn 


off 


the 


road 



When all possible parses have been stored in a chart a Vitcrbi->i;-te search algorithm can locate the most 
probable partial parse. In the above example, this would hopefully show that Ihe parse (turn) (off (die 
road)) is more probable than any of (turn off) (the road), ((urn ) off (the road), (turn) off (he road, turn off 
(the road), turn i(nff) the road) or the **zero" parse turn off the r.\id. This raises an interesting situation: 
since a chart can be thought of as a lattice, and so non-dctcrmimfiic parsing can be seen as taking an 



SOOCIO. <WO_03C01S04A1_L> 



WO 03/001504 PCT/AU02/00802 

16 

ft*- A3 



3/5 



utterance represented as a simple lattice where each arc represents a single word and adding arcs 
corresponding to phrasal categories. This creates a lattice of all partial parses. 



VERB PP 




Furthermore, the input to this need not be a single utterance, but can be a lattice of recognition hypotheses. 
For example, the lattice below shows the results of non-deterministically parsing a recognition lattice which 
initially contained just the two traversal: turn off (he road and turn off the light (inloduced arcs are shown 
in dotted line). 



Pf: off the road 




Once this lattice has been constructed, the Vitcrbl search should produce one offruni off) (the light) and (turn) (off (the 
road)) as the most likely parse. 



To summarise: 

• nondeterministic parsing is done most efficiently using ch2n parsing; 

• chart parsing is equivalent lo adding arcs to a lattice; 

hence parsing of recognition hypothesis lattices is a natural extension lo parsing single utterances. 

Non-deterministic partial parsing and maxima! parses 

The above discussion holds for both full and partial parsing. In roth cases the chart/lattice containing all 
parsed structures is constructed. The only difference is that a full parse requires the existence of an arc 
leading straight from the START node of the lattice to the end NODE. Probabilistic full parsing requires 
selecting the most probable of all such arcs. Probabilistic pnni3i parsing requires a Vilerbi search through 
the lattice to find the most probable path from the START node to the END node. Depending on the 
modelling techniques and the available statistics, it is theoretically possible that the Vitcrbi search may 
produce a partial parse which is just a "zero" parse, i:e. just a sequence of words when in fact a phrasal arc 
was present in the parse lattice. One way to avoid this is lo onlv illow partial parses which are maximal. 
By maximal, i mean lhai there exists no other parse which is idenucal except for the application of another 
rule. Perhaps the easiest way to preclude the Vilerbi search from returning non-maximal parses is to 
cleverly prune ihe lattice of arcs in such a way that all travcrsals of the lattice produce maximal parses. 
Care must be taken when pruning in order noi lo exclude any maximal parses. Reflection led mc to the 
following pruning procedure for achieving this. In fact, it may be that the properly of being included in a 
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maximal parse is not sufficiently local for an elegant pruning algorithm to exist. So in the meantime we 
must trust that the training data will never prefer a non-maximal parse. 

Probabilistic partial parsing - the nitty gritty 

Unable to find any established results in the literature on probabilistic partial parsing, in this section I 
derive the formulas required from scratch. Recall that the Inside and Outside probabilities are used in 
probabilistic parsing. The Inside probability of a subtree is the probability of all the expansions within that 
subtree, whereas the Outside probability is the probability of that subtree ocurring within the context of the 
sentence. In the case of complete parses, the supert^ees , phrase expansion probabilities are used to 
calculate the Outside probabilities. However since we are guaranteed only partial parses, we often will not 
have a supcrtree. In these cases we shall use category-based N-gram language modelling to calculate the 
Outside probabilities, where the categories can be either phrasal or lexical. 

So, assume that a partial parse produces a sequence of categories Q. The probability of the parse is 
calculated by: 



Where P Q m is calculated using (backoff) Ingram probabilities, and P t „ is the inside probability of the 
category, which in the case of lexical categories defaults to the probability of the word given the word 
class. Note that under this model probabilistic parsing replace? language modelling. However it would 
also be possible to do some language modelling earlier on in order to reduce the candidate space, then 
ignore the language modelling results and do probabilistic parsing. 

In practise, the outside probabilities can be calculated using the Acoustic group's language model class. 
This means we must just calculate the inside probabilities, which is quite a simple taskof multiplying 
probabilities of phrase expansions and lexical instantiations. Note also that it will also be easy to do speech 
repair by doing lattice pruning, and also ALARM processing is a simple case of lattice pruning. This 
shows we are ready to accept lattices from the acoustics group. Even more, if we decide to proceed with a 
non-deterministic partial parser then lattice parsing is not just p-rtStbic but desirable! 

Data, damned data! 

In order to calculaie the inside probabilities of phrase trees, and also the outside probabilities using N-gram 
probabilities of phrasal categories, a corpus of parsed sentences must be available. I propose thai we 
develop a semi-automatic procedure whereby the user is preserved with all maximal parses of each 
utterance in the corpus. They then make choose the correct one. and this information results in frequency 
statistics for N-grams and phrase structure expansions. 

A non-staftslfcaf approach to disambiguation 

Since we will not always have parsed corpora, the analyser sh«««.J be able to disambiguate using 
information from the Meiagrainmar. Here I propose a non-dcirr-ninistic method of doing analysis lor 
speech recognition applications. Crucial to the proposal is that ~rf grammar for analysis should be derived 
from a speech recognition grammar in which certain chunks, m rhrases. have been indicated by the 
grammar to be meaningful. 

Advantages: 

• Automatically and efficiently disambiguates problem phra>i> 

• Eliminates the need for condi lions in the grammar 

• Eliminates the dependence on rule order 

Before describing the algorithm. I first introduce the concept of; >i> chunks. 
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• By top chunk I mean a rule of the speech recognition grammar that has semantic tags, but all of the 
rules that' reference do not have semantic tags. 

During analyser compilation: 

1) Find each top chunk of the grammar. 
During analysis: 

• > 

2) Use'chunk parsing as we currently do, except multiple hypotheses can be kept alive at each level. 

3) After parsing, discard all hypotheses that are not top-level rules. These are cases that have already 
been disambiguated from context 

4) If ambiguities remain (i.e. if there exist overlapping top chunks), resolve them using the recognition 
grammar. 
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CLAIMS 

1 . In a speech recognition system of the type adapted to 
process utterances from a caller or user by way of 
components including a recogniser, an utterance 
5 processing system and a dialogue processing system so 

as to produce responses to said utterances, a method 
of parsing a data structure derived by a component of 
said system; said method comprising performing only a 
partial parse of said data structure. 

10 2. In a speech recognition system of the type adapted to 
process utterances from a caller or user by way of 
components including a recogniser, an utterance 
processing system and a dialogue processing system so 
as to produce responses to said utterances, a 

15 stochastic chunk parser for attribute grammars which 

searches for meaningful words and phrases within said 
utterances and from these builds attribute value 

r 

matrices . 

3. The parser of Claim 2 wherein said parser utilizes 
20 structure and location of said phrases to estimate the 

probability of the utterance in which they are found. 

4 . The parser of Claim 2 or Claim 3 which receives as 
input an N-best list derived from said recogniser. 
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5 . The parser of any previous claim wherein said 
attribute value matrices attach respective 
probabilities to respective attribute value matrices 
by combining phrase structure probabilities with 

5 class-based N-gram language modeling. 

6. The method of . Claim 1 applied to a data structure. 

7. The method of Claim 6 wherein said data structure 
comprises a N-best list. 

8. The method of Claim 6 wherein said data structure 
10 comprises a word lattice. 
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